Evolutionary design of deep neural networks

Mención Internacional en el título de doctorFor three decades, neuroevolution has applied evolutionary computation to the optimization of the topology of artificial neural networks, with most works focusing on very simple architectures. However, times have changed, and nowadays convolutional neural networks are the industry and academia standard for solving a variety of problems, many of which remained unsolved before the discovery of this kind of networks. Convolutional neural networks involve complex topologies, and the manual design of these topologies for solving a problem at hand is expensive and inefficient. In this thesis, our aim is to use neuroevolution in order to evolve the architecture of convolutional neural networks. To do so, we have decided to try two different techniques: genetic algorithms and grammatical evolution. We have implemented a niching scheme for preserving the genetic diversity, in order to ease the construction of ensembles of neural networks. These techniques have been validated against the MNIST database for handwritten digit recognition, achieving a test error rate of 0.28%, and the OPPORTUNITY data set for human activity recognition, attaining an F1 score of 0.9275. Both results have proven very competitive when compared with the state of the art. Also, in all cases, ensembles have proven to perform better than individual models. Later, the topologies learned for MNIST were tested on EMNIST, a database recently introduced in 2017, which includes more samples and a set of letters for character recognition. Results have shown that the topologies optimized for MNIST perform well on EMNIST, proving that architectures can be reused across domains with similar characteristics. In summary, neuroevolution is an effective approach for automatically designing topologies for convolutional neural networks. However, it still remains as an unexplored field due to hardware limitations. Current advances, however, should constitute the fuel that empowers the emergence of this field, and further research should start as of today.This Ph.D. dissertation has been partially supported by the Spanish Ministry of Education, Culture and Sports under FPU fellowship with identifier FPU13/03917. This research stay has been partially co-funded by the Spanish Ministry of Education, Culture and Sports under FPU short stay grant with identifier EST15/00260.Programa Oficial de Doctorado en Ciencia y Tecnología InformáticaPresidente: María Araceli Sanchís de Miguel.- Secretario: Francisco Javier Segovia Pérez.- Vocal: Simon Luca

Editor’s Note. Towards Blockchain Intelligence

In this special issue, we want to gather some innovative applications that are currently pushing forward the research on Blockchain technologies. In particular, we are interested also in those applications that put the focus on the data, enabling new processes that are able to leverage relevant knowledge from the data. This special issue will be successful if readers gain a better understanding on how Blockchain can be applied to very diverse areas, and might even be interested in designing, implementing and deploying an innovative solution to a completely different field of knowledge. We hope this Special Issue can provide a better understanding and key insights to readers on how Blockchain and artificial intelligence are cross-fertilizing to revolutionize many aspects in our societies

Editor’s Note. Towards an Intelligent Society: Advances in Marketing and Neuroscience

This Special Issue focuses in cases that explore the relationship between Artificial Intelligence and marketing, as well as neuroscience. AI can be combined with specific neuroscience techniques to achieve a more successful and profitable neuromarketing. For this Special Issue, we have found that descriptions of successful use cases are highly valuable to help researchers identify fields where novel applications of AI can enhance the outcome of digital marketing and neuroscience

AWS PredSpot: Machine Learning for Predicting the Price of Spot Instances in AWS Cloud

Elastic Cloud Compute (EC2) is one of the most well-known services provided by Amazon for provisioning cloud computing resources, also known as instances. Besides the classical on-demand scheme, where users purchase compute capacity at a fixed cost, EC2 supports so-called spot instances, which are offered following a bidding scheme, where users can save up to 90% of the cost of the on-demand instance. EC2 spot instances can be a useful alternative for attaining an important reduction in infrastructure cost, but designing bidding policies can be a difficult task, since bidding under their cost will either prevent users from provisioning instances or losing those that they already own. Towards this extent, accurate forecasting of spot instance prices can be of an outstanding interest for designing working bidding policies. In this paper, we propose the use of different machine learning techniques to estimate the future price of EC2 spot instances. These include linear, ridge and lasso regressions, multilayer perceptrons, K-nearest neighbors, extra trees and random forests. The obtained performance varies significantly between instances types, and root mean squared errors ranges between values very close to zero up to values over 60 in some of the most expensive instances. Still, we can see that for most of the instances, forecasting performance is remarkably good, encouraging further research in this field of study

Circumstancial knowledge management for human-like interaction

This project focuses on the circumstantial aspects of human-like interaction. Its purpose is in the first place to design and develop a general-purpose situation model, which would enhance any natural interaction system by providing knowledge on the interaction context, and secondly to implement an intuitive management tool to edit the knowledge base of this situation model. The development of both the model and the edition tool has followed the usual processes of software engineering: requirements elicitation and specification, problem analysis, design of a solution, system implementation and validation. In more specific terms, an spiral lifecyle composed of three phases was followed, which is a convenient approach in research projects as all the system requirements might not be known from the very beginning. After the implementation was completed, an evaluation was carried out for observing the advantages of the edition tool over the manual edition of the model knowledge. The results of this evaluation showed that the tool provides a mechanism for the model management which is significantly faster and more accurate than the manual edition. Moreover, a subjective survey also revealed that the experiment subjects preferred the edition tool, as they considered it to be more comfortable, more intuitive, more reliable and more agile. The resulting general-purpose situation model and the edition tool are a significant contribution to the state of the art, as the previously existing situation models were ad-hoc models, i.e., models implemented for supporting an specific system, and their knowledge bases were edited manually. Finally, this work will be applied in a research project funded by the Spanish Ministry of Industry (CADOOH, TSI-020302-2011-21), and for this reason some of the future works observed in this document will be executed in the coming months. A copy of the Cognos toolkit, including the edition tool developed within this project, is available in the next site: http://labda.inf.uc3m.es/doku.php?id=es:labda_lineas:cognos.Ingeniería Informátic

Editor's Note. Towards an intelligent society: advances in marketing and neuroscience

Special Issue on Use Cases of Artificial Intelligence, Digital Marketing and Neuroscience.This Special Issue focuses in cases that explore the relationship between Artificial Intelligence and marketing, as well as neuroscience. AI can be combined with specific neuroscience techniques to achieve a more successful and profitable neuromarketing. For this Special Issue, we have found that descriptions of successful use cases are highly valuable to help researchers identify fields where novel applications of AI can enhance the outcome of digital marketing and neuroscience

DataCare: Big Data Analytics Solution for Intelligent Healthcare Management

This paper presents DataCare, a solution for intelligent healthcare management. This product is able not only to retrieve and aggregate data from different key performance indicators in healthcare centers, but also to estimate future values for these key performance indicators and, as a result, fire early alerts when undesirable values are about to occur or provide recommendations to improve the quality of service. DataCare’s core processes are built over a free and open-source cross-platform document-oriented database (MongoDB), and Apache Spark, an open-source cluster-computing framework. This architecture ensures high scalability capable of processing very high data volumes coming at fast speed from a large set of sources. This article describes the architecture designed for this project and the results obtained after conducting a pilot in a healthcare center. Useful conclusions have been drawn regarding how key performance indicators change based on different situations, and how they affect patients’ satisfaction

Feature set optimization for physical activity recognition using genetic algorithms

Proceeding of: Genetic and Evolutionary Computation Conference (GECCO 2015)Physical activity is recognized as one of the key factors for a healthy life due to its beneficial effects. The range of physical activities is very broad, and not all of them require the same effort to be performed nor have the same effects on health. For this reason, automatically recognizing the physical activity performed by a user (or patient) turns out to be an interesting research field, mainly because of two reasons: (1) it increases personal awareness about the activity being performed and its consequences on health, allowing to receive proper credit (e.g. social recognition) for the effort; and (2) it allows doctors to perform continuous remote patient monitoring. This paper proposes a new approach for improving activity recognition by describing an activity recognition chain (ARC) that is optimized by means of genetic algorithms.This optimization process determines the most suitable and informative set of features that turns out into higher recognition accuracy while reducing the total number of sensors required to track the user activity. These improvements can be translated into lower costs in hardware and less intrusive devices for the patients. In this work, for the assessment of the proposed approach versus other techniques and for replication purposes, a publicly available dataset on physical activity (PAMAP2) has been used. Experiments are designed and conducted to evaluate the proposed ARC by using leave-one-subject-out cross validation and results are encouraging, reaching an average classification accuracy of about 94%.This research was partially funded by European Union’s CIP Programme (ICT-PSP-2012) under grant agreement no. 325146 (SEACW project)

A survey of handwritten character recognition with MNIST and EMNIST

This article belongs to the Special Issue Computer Vision and Pattern Recognition in the Era of Deep Learning.This paper summarizes the top state-of-the-art contributions reported on the MNIST dataset for handwritten digit recognition. This dataset has been extensively used to validate novel techniques in computer vision, and in recent years, many authors have explored the performance of convolutional neural networks (CNNs) and other deep learning techniques over this dataset. To the best of our knowledge, this paper is the first exhaustive and updated review of this dataset; there are some online rankings, but they are outdated, and most published papers survey only closely related works, omitting most of the literature. This paper makes a distinction between those works using some kind of data augmentation and works using the original dataset out-of-the-box. Also, works using CNNs are reported separately; as they are becoming the state-of-the-art approach for solving this problem. Nowadays, a significant amount of works have attained a test error rate smaller than 1% on this dataset; which is becoming non-challenging. By mid-2017, a new dataset was introduced: EMNIST, which involves both digits and letters, with a larger amount of data acquired from a database different than MNIST's. In this paper, EMNIST is explained and some results are surveyed

Evolutionary Design of Convolutional Neural Networks for Human Activity Recognition in Sensor-Rich Environments

Human activity recognition is a challenging problem for context-aware systems and applications. It is gaining interest due to the ubiquity of different sensor sources, wearable smart objects, ambient sensors, etc. This task is usually approached as a supervised machine learning problem, where a label is to be predicted given some input data, such as the signals retrieved from different sensors. For tackling the human activity recognition problem in sensor network environments, in this paper we propose the use of deep learning (convolutional neural networks) to perform activity recognition using the publicly available OPPORTUNITY dataset. Instead of manually choosing a suitable topology, we will let an evolutionary algorithm design the optimal topology in order to maximize the classification F1 score. After that, we will also explore the performance of committees of the models resulting from the evolutionary process. Results analysis indicates that the proposed model was able to perform activity recognition within a heterogeneous sensor network environment, achieving very high accuracies when tested with new sensor data. Based on all conducted experiments, the proposed neuroevolutionary system has proved to be able to systematically find a classification model which is capable of outperforming previous results reported in the state-of-the-art, showing that this approach is useful and improves upon previously manually-designed architectures.This research is partially supported by the Spanish Ministry of Education, Culture and Sports under FPU fellowship with identifier FPU13/03917